<19) 





(12) 



(43) Date of publication: 

06.10.1999 Bulletin 1999/40 

(21) Application number: 99302131.0 

(22) Date of filing: 19.03.1999 



EuropSisches Patentamt 
European Patent Office 
Office europ£en des brevets (11) EP 0 947 925 A2 

EUROPEAN PATENT APPLICATION 

(51) Int a. 6 : G06F 9/46, G06F 1/00 



(84) Designated Contracting States: 

AT BE CH CY DE DK ES H FR GB GR IE IT LI LU 
MCNLPTSE 

Designated Extension States: 
ALLTLVMKROSI 

(30) Priority: 01.04.1998 US 53571 

(71) Applicant: 

Hewlett-Packard Company 
Palo Alto, California 94304 (US) 



(72) Inventor: Campbell, Randall B. 
Fort Collins, CO 80525 (US) 

(74) Representative: 

Coigan, Stephen James et al 
CARPMAELS & RANSFORD 
43 Bloomsbury Square 
London WC1 A 2RA(GB) 



CM 
< 

in 

CM 

ay 



(54) Apparatus and method for remotely executing commands using distributed computing 
environment remote procedure calls 



(57) The present invention generally relates to an 
apparatus and method of providing security for remote 
command execution. Remote command execution is a 
process where a local host processor (11) causes a pro- 
gram to be executed on a remote host processor (12). 
The method of security provides for dynamically adapt- 
ing the security methods in a distributed computing 
environment communicating using remote procedure 
calls (RPCs) across a network. The method includes 
the step of sensing if DCE security or a default security 
method is to be utilized. 
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Description 

BACKGROUND HF THE INVENTION 
FIELD OF TH E INVENTION 

[0001 ] "me present invention generally relates to com- 
puters and software, and more particularly, to the secu- 
rity involved in remote command execution. Remote 
command execution is a process where a local host 
processor causes a program to be executed on a 
remote host processor. Usually, the calling process 
passes data to the remote program and captures the 
output generated by the remote program on the remote 
host 

DESCRIPTIO N OF RELATED ART 

[0002] As known in the computer and software arts, a 
remote procedure call (RPC) is equivalent to a call to a 
local subroutine. An application program that makes the 
call executes the call statement, passes the parameters 
to the called procedure, and then waits for the results to 
be passed back from the called procedure. However, 
unlike a local procedure call, the procedure called by 
the RPC does not reside in the application program or 
may not even reside in the computer running the appli- 
cation program, but instead resides on a remote compu- 
ter in a network. It is also known that an RPC can be 
used for remote command execution. 
[0003] In unix computing, the existing art for executing 
commands remotely consists of the Berkely Software 
Distribution Unix (BSD) 4.3 family of commands and 
procedure calls (rsh (remsh). rcmd. rexec) and the 
RPC-based on/rexd combination developed by Sun 
Microsystems, Inc. 

[0004] The BSD facilities rely on a programming inter- 
face known as "sockets," wherein a two-way connection 
is created between the client and the server. "On" and 
"Rexd" are based on remote procedure calls, which is a 
programming interface wherein procedure calls are 
made transparently on the client, but execute on the 
server. 

[0005] In all cases, a "daemon" program running per- 
manently on the server system has the responsibility of 
creating or denying the connection from the client appli- 
cation to the service provider; or server application. 
[0006] A key difficulty with remote command execution 
is that of security. How does one ensure that only 
allowed users from allowed client systems are permitted 
access to the server application, and also ensure that 
access cannot be perverted to malicious purposes? 
[0007] The method used in BSD 4.3 unix works as fol- 
lows. (1) The server checks the client's TCP port (an 
address associated with a socket to ensure that it is a 
reserved port (2) The server reads the following inputs 
from the client: the client-side user name, the requested 
server-side user name, and the command to be exe- 



cuted. (3) The requested user name on the server is 
looked up in the password database on the server sys- 
tem. If there is no entry, then the connection is termi- 
nated. (4) One of two authorization files 
5 (/etc/hosts, equiv. and the .rhosts in the user's home 
directory) are checked for the client host name and the 
client user name. 

[0008] If all these checks pass, execution of the com- 
mand is allowed. 

io [0009] The RPC-based remote execution facility pro- 
duced by Sun Microsystems, Inc. has inherently almost 
no security - if the same user ID exists on the server as 
on the client, execution is allowed. The one significant 
restriction is that no remote execution is allowed by root. 

is There is a command line option to rexd that causes it to 
rely on the BSD 4.3 security mechanism described 
above. 

[0010] The prior solutions for remote command execu- 
tion suffer from the following security weaknesses. It will 

20 be shown later how the present invention addresses 
and overcomes certain of these difficulties. A problem 
with the prior solutions is that the host addresses and 
user names are sent in plain text that is very open to 
"spoofing". A knowledgeable hacker can transmit pack- 

25 ets pretending to be from another machine or another 
user. 

[0011] Another problem is that the authentication is 
weak, because the server accepts the user and host 
name as identified in the transmission without proof. 

30 [0012] Yet another problem arises in cases where 
passwords are required. They are transmitted over the 
network in clear text, which is subject to interception. 
[001 3] Another problem is that the authorization files 
not very well protected. The files .rhosts and 

35 /etc/hosts.equiv are used to circumvent sending pass- 
words, but they are not well protected. The 
/etc/hosts.equiv file is generally write-protected to root 
only, but individual users' .rhosts are often not well-pro- 
tected, and it is hard to enforce strong protection. They 

40 can be modified to allow unintended users to execute 
commands. 

[001 4] Furthermore, there is the problem that no state 
is maintained since each command transaction stands 
alone. This leaves these methods open to "replay 

45 attacks" wherein a hacker captures a valid network 
packet, alters some details (like the name of the user, or 
the command to execute) and resends it. 
[001 5] In addition, there is the problem that there are 
few options for additional security. The file 

so /var/adm/inetd.sec can be used to limit accessibility of 
the services to a specific set of hosts. There are no 
options to encrypt data or obtain stronger authentica- 
tion. There are additional levels of security that can be 
irrposed on unix systems at the operating system level. 

55 These are special versions of unix, usually developed 
independently by vendors to meet the government B2 
Security requirement. These are outside the scope of 
the present invention, as they require a special version 
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of unix. 

[001 6] Another method of imposing stronger security 
on unix operations with modifying the base operating 
system is the Kerberos system. Kerberos uses a "third 
party," a separate computer designated as the security s 
server. All "principals" in the distributed network must be 
known to the security server, and all have passwords, or 
keys, known only to themselves and the security server. . 
[0017] Each principal (which may be a user or a pro- 
gram) must obtain a set of credentials, known as a 10 
ticket, from the security server before it may perform any 
functions over the network. In addition, before a client 
can connect to a server, it must obtain a different, spe- 
cific ticket from the security server which it presents to 
the server. All keys and tickets are encrypted in such a is 
way that only the intended recipient can decrypt them. 
[0018] However, until now. network systems have 
lacked the ability to provide flexible and heightened 
security for remote procedure calls in the distributed 
computing environment (DCE). 20 

SUMMARY OF THE INVENTION 

[001 9] Certain objects, advantages and novel features 
of the invention will be set forth in part in the description 25 
that follows and in part will become apparent to those 
skilled in the art upon examination of the following or 
may be learned with the practice of the invention. The 
objects and advantages of the invention may be realized 
and obtained by means of the instrumentalities and 30 
combinations particularly pointed out in the appended 
claims. 

[0020] To achieve the advantages and novel features, 
the present invention is generally directed to a method 
for remotely executing commands using remote proce- 35 
dure calls (RPCs) in the distributed computing environ- 
ment (DCE). In accordance with one aspect of the 
invention, an apparatus and method dynamically adapt 
the security methods in a distributed computing environ- 
ment communicating using remote procedure calls 40 
(RPCs) across a network. The apparatus and method 
irrplement the function of sensing if DCE security or a 
default security method is to be utilized. 
[0021 ] In accordance with the one embodiment of the 
method of the present invention, enhanced security is 45 
afforded by providing the host identification and user 
name in RPC packet in binary form. These data items 
can still be extracted, but not as easily. 
[0022] In accordance with another embodiment of the 
present invention, authentication is strengthened, even so 
in the default security mode by use of a shared secret 
key and in the enhanced security mode by using the 
very strong Kerberos-based authentication. 
[0023] In accordance with another embodiment of the 
present invention, a single system-level authorization 55 
file is root-protected, and limits client access to a single 
system. 

[0024] In accordance with another embodiment of the 



present invention, the default security mode, a non- 
repeating value field in the security structure is also 
encoded with the secret key. This non-repeating value 
field effectively prevents replay attacks. In the enhanced 
security mode, the Kerberos ticketing system performs 
this function. 

[0025] An alternate embodiment provides an appara- 
tus and method for implementing stronger security in 
the default mode than the prior art, but can go beyond 
that by sensing its environment, and if in a properly con- 
figured DCE cell, makes use of Kerberos-style third- 
party authentication and authorization, and authenti- 
cates every packet. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0026] The accompanying drawings incorporated in 
and forming a part of the specification illustrate several 
aspects of the present invention, and together with the 
description, serve to explain the principles of the inven- 
tion. In the drawings: 

FIG. 1 is a block diagram of the client/server system 
utilizing the 4.3 BSD security system of the prior 
art. 

FIG. 2 is a block diagram of the client/server system 
utilizing the Kerberos security system of the prior 
art. 

FIG. 3 is a block diagram of the client/server system 
utilizing the dynamic security system and method of 
the present invention. 

FIG. 4 is a flow chart of the process for the default 

security system operating in the client system of the 

present invention, as shown in FIG. 3. 

FIG. 5 is a flow chart of the process for the default 

security system operating in the server system of 

the present invention, shown in FIG. 3. 

FIG. 6 is a flow chart of the process for the DCE 

security system operating in the client system of the 

present invention, shown in FIG. 3. 

FIG. 7 is a flow chart of the process for the DCE 

security system operating in the server system of 

the present invention, shown in FIG. 3. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

[0027] Reference will now be made in detail to the 
description of the invention as illustrated in the draw- 
ings. While the invention will be described in connection 
with these drawings, there is no intent to limit it to the 
embodiment or embodiments disclosed therein. On the 
contrary, the intent is to cover all alternatives, modifica- 
tions, and equivalents included within the spirit and 
scope of the invention as defined by the appended 
claims. 

[0028] Turning now to the drawings. FIG. 1 is a block 
diagram of the client server system utilizing the 4.3 BSD 
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security system of the prior art. Client 1 1 comprises an 
application 21 that requests server functionality utilizing 
the communication link 22 to the network interface 23 
that communicates with server 1 2. 
[0029] The 4.3 BSD routines running in a unix environ- 
ment generally require a host to prove its identity and 
trusts the host identification of a user on that host. The 
hosts are identified by their Internet addresses and 
trusted hosts are assumed to identify correctly the user 
on that host requesting a particular service. The 4.3 
BSD security scheme uses a concept of reserved Inter- 
net ports wherein the remote procedure binds with a 
reserved port However, there is nothing to stop any 
user from knowing the root password on a station from 
binding to a reserved Internet port. 
[0030] The authentications based upon the Internet 
address includes both the 32 bit Internet network identi- 
fication (ID) and the host ID as well as the 16-bit TCP 
port number. The steps for the server security include: 
(1 ) The check of the client's TCP port to verify that it is a 
reserved port on link 1 4. This port corresponds to a con- 
nection in the socket. If not an authorized socket, the 
connection is terminated. (2) The server reads the fol- 
lowing three strings from the client user name, the 
server user name, and the command string to be exe- 
cuted on the remote host in the server authentication 
functionality 31. (3) The server user name is looked up 
in the password file by the server authentication func- 
tionality 31. H no entry is found, then the connection is 
terminated. (4) If the password file entry does contain a 
password, the client host name and the server user 
name are verified by searching a standard authorization 
file 3 1 on the server. Standard authorization files are not 
encrypted. 

[0031] The application 21 accomplishes the request 
for server functionality by the use of a programming 
interface known as sockets. The application 21 writes its 
data to the client side of the socket connection 16. The 
service provider reads the data from its end, and writes 
its output on its end of the socket connection 1 7. 
[0032] The network interface determines which net- 
work transport (i.e., TCP/IP or UDP/IP) to use for the 
communication, and searches the directory services for 
the server's host directory and then connects to and 
transmits through a socket to the server over link 16. 
[0033] First, the client sends a message across line 1 4 
to the server 12 service authentication module 31. The 
server 12 service authentication module 31 checks the 
authentication of the client base. Next, the client sends 
the data link on 16 requesting the particular service to 
be performed. The server performs the requested serv- 
ice and return the results 1 7 to the client requesting the 
service. 

[0034] Illustrated in FIG. 2 is a block diagram of the cli- 
ent server system utilizing the Kerberos security system 
of the prior art. Client 11 is comprises an application 
program 21 communication link 22 to a DCE interface 
23. The steps performed to execute a remote procedure 



call in a Kerberos security system are as follows. First, 
the user logs into a client workstation by entering their 
login name at the unix login prompt. 
[0035] Next, as a part of the login sequence and 

5 before prompted for the password, the message 24 is 
sent across the network to the Kerberos authentication 
security server 13. This message 24 contains the user 
login name along with the namepf one particular Ker- 
beros server. Since the message contains only two 

10 names, it need not be encrypted The names are not 
considered secrets and everyone has to know the other 
names of client and servers to communicate. 
[0036] Then, the authentication security server 13 
looks up the user login name and the service name in 

15 the Kerberos database and obtains an encryption key 
for each. The encryption keys used by the Kerberos are 
one-way encrypted passwords similar to what is stored 
in the password entry field of a normal unix password 
file. The authentication security server 13 forms a 

20 response 44 back to the client login program on the 
workstation. This response contains a ticket that grants 
the user access to the requested server 12. The con- 
cept of the ticket and what makes up the ticket is the 
core of the Kerberos system. Tickets are always sent 

25 across the network in an encrypted form. They are 
never sent in clear text. The messages and the ticket is 
encrypted using the client's encryption key and encryp- 
tion password which is contained in the Kerberos data- 
base on security server 13 in the service authentication 

30 module 42. 

[0037] Next, the client login program receives the 
enrypted message 44 and only then prompts the user 
for their password. This clear text password is first proc- 
essed in the standard unix one-way encryption algo- 

35 rithm and as a result is used to encrypt the ticket 
message 44 received. Then, the clear text password is 
erased from memory leaving a sealed ticket that is 
encrypted with the session key. At this point, the client 

1 1 software saves the copy of the ticket and the session 
40 key. To request a specific service, the client 1 1 must 

obtain a ticket for the particular service. To obtain a 
ticket, the client 1 1 software has to contact the Kerberos 
ticket granting server 13. 

[0038] Then, the workstation builds a message 25 to 
45 be sent to the ticket granting service. This message 
consists of a sealed ticket, a sealed authenticator and 
an inserver name. The sealed authenticator includes 
the client workstation login name, the workstation net 
address, and the current time. This message is then 
so sent to the ticket granting service and the server 12. 
[0039] Next, the ticket granting service of the server 

1 2 receives the message 25 and encrypts the message 
using the encryption key. From the unencrypted mes- 
sage the server 12 obtains the session key. This ses- 

55 sion key is used to encrypt the ticket from the sealed 
authenticator. There are multiple items for the ticket 
granting service to check for validity, the login name of 
both the ticket and the authenticator, and the server 
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name in the ticket. The server also compares the net- 
work address of the ticket, the authenticated and the 
received message. The server 1 2 then looks up the end 
server name from the message in the Kerberos data- 
base and obtains the encryption key for the specified s 
service. 

[0040] Then, the ticket granting service forms a new 
. random session key and then creates a new ticket 
based on the requested end service name and the new 
session key. The ticket is sealed using the encryption 10 
key for the requested end server. It is then sent to the 
workstation. 

[0041] Soon afterward, the workstation receives the 
sealed message 35, decrypts it using the session key 
that it knows. From this message it again receives a is 
sealed ticket that it cannot decrypt. This sealed ticket is 
what it has to send to the end service server. 
[0042] Then, the client workstation builds an authenti- 
cator including the login name, workstation, network 
address, and the current time and seals it using the new 20 
session key. Finally, the client workstation is able to 
send a message 16 to the end service server. This mes- 
sage is not encrypted since both the ticket and the 
authenticator within the message are sealed and the 
name of the end server is not a secret 25 
[0043] Last, the end service server receives this mes- 
sage and decrypts the sealed ticket using its encryption 
key which only the end server and the Kerberos security 
server know. The end service server 12 then uses the 
new session key contained in the ticket to encrypt the 30 
authenticator and do the requested service processing. 
The end service server 12 sends the results of the 
requested service processing in message 1 7 back to 
the client. 

[0044] Illustrated in FIG. 3 is the security system of the 35 
present invention. A message from application program 
21 sent across communication link 22 creates a client 
code write remote stub 27. "This client code write remote 
stub first checks to see if the client security is set to DCE 
security, rf the client code is set to DCE security, then 40 
the authorization is set to DCE. Otherwise, the authori- 
zation is set to the default security system. 
[0045] Then, while input exists from the client 1 1 appli- 
cation 21 , the client code write remote stub processes a 
loop that first checks to see if the authorization is set to 4s 
DCE. If the security is set to DCE, then an authorization 
packet is sent across in 36. The next step in the loop is 
to put the input received from this stdin 26a into a buffer 
and perform a remote send (i.e.. a remote procedure 
call) 36 to the server 12. so 
[0046] The server 12 code stub program 38, upon ini- 
tialization checks to see if security is set to DCE, then if 
so, registers 34 with the security server 13 (in FIG. 2). 
Then the server code stub 38 waits for a call from a cli- 
ent code sub write remote stub program. Once a remote ss 
send RPC is received, the server code 30 program 38 
forks and executes the remote send and remote receive 
routines in parallel. 



[0047] The remote send procedure first checks if data 
received is the first session. If so. then remote send sets 
up its communication pipes. Next the remote send sub 
routine remote thread forks and executes subproce- 
dures to perform the actual service. The remote send 
sub procedure then sends the data received in the 
buffer from the client code write remote stub 27 across 
line 36 to the appropriate sub procedures stdin via the 
pipe created in i he initialization step. 
[0048] The remote receive stub program, upon execu- 
tion, reads the stdout via pipe from the sub procedures 
executed in the remote send. The remote receive stub 
puts the data from the sub procedures into a buffer 
which is then returned to the client caller remote proce- 
dure call across line 39. The client code read remote 
stub program 28 after initialization is suspended until 
receiving the first turn of data from the server code 
remote receive program. Then, while the client code 28 
read remote 28 receives data from the server, it 
receives the data from the remote procedure call and it 
writes the data in the buffer from the remote procedure 
call to the stdout for return to the client application pro- 
gram 21 . The stdout data is sent via line 26b. 
[0049] Illustrated in FIG. 4 is the process for the client 
system security initialization and security process for 
remote procedure calls in more detail with regard to the 
selection of default security. 

[0050] As the client system is initialized in step 51 , this 
includes operating system and all other initialization 
maintenance program loading. Step 51 also includes 
the execution of the client application program 21 (FIG. 
1). 

[0051] Upon the client application program 21 execu- 
tion of a remote procedure call results in the initialization 
of the remote execution client (referred to below as 
"rem ex") which is one per session in step 52. The 
remote execution client ("remex") performs the following 
actions, reads its stdin, which is being fed by the client 
application program, writes what it reads to the remote 
execution server by remote procedure calls, receives 
the output from the remote execution server by another 
RPC, and writes the output to its stdout, where it is 
picked up by the client application program. Remex 
essentially functions as a data pipeline; the data being 
passed back and forth is opaque to it, with the exception 
of internally-generated error messages. 
[0052] After initialization there is a check to see if the 
security for the remote procedure call is DCE. This is 
accomplished by the application checking, via a DCE 
API, whether it's in a configured DCE cell. If so, it then 
attempts to get a ticket using an internally-derived prin- 
cipal name, ff these checks succeed, and it is able to 
verify its identity with the Security Server, then the DCE 
security method is utilized and the process uses the 
flowchart in FIG. 6. Otherwise, the authorization is set to 
the default security method at step 53. The default 
security method is accomplished by creating an 
encoded key by concatenating a password, shared with 
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each of the remote clients, with a pseudo-random 
number. The resulting string is then encrypted using a 
library call. The first two characters of the encrypted 
result (which contain the "seed" used to generate the 
encryption) are stripped off. The password is stored s 
internally in an encoded format (as an array of shorts, 
with the high bit of the character value set), to make it 
hard to read from the executable. 
[0053] This encrypted toy, the pseudo-random 
number, the user name, and the hostname (of the 10 
server) are all placed in fields in a custom binding han- 
dle that is used to bind with the remote. The security 
check is done when binding takes place at the begin- 
ning of a session by the first outgoing RPC. Subsequent 
calls within the same session are not re-authenticated, is 
[0054] Next, the write and read remote threads are 
created and executed at step 54. The write remote and 
read remote threads execute in parallel and after com- 
pletion of the remote procedure call, each of the write 
remote and read remote threads are terminated. 20 
[0055] The write remote thread, upon execution, first 
suspends the read remote thread and binds to the 
remexd daemon of the server system at step 55. Next, 
the write remote thread sets up the security structure for 
the default security being used for the remote procedure 2s 
call and reads from stdin at step 56. The write remote 
thread performs a remote procedure call to rem-send in 
the server 12 system (FIG. 1) and unsuspends the read 
remote thread at step 57. At this time, the read remote 
thread is unsuspended and processes and this will be so 
described in later paragraphs. 

[0056] The write remote thread performs reading of 
the stdin file and remote procedure calling of the rem- 
send until input from stdin file is empty at step 58. Once 
input from standard in file is empty, then the write 35 
remote thread exits at step 59. 

[0057] Upon execution of the read remote thread, the 
read remote thread suspends until the first rem-send 
call is sent to the server. This is done because until 
there is a remote procedure call to the server, there can 40 
be no returning of data from this server based on the 
remote procedure call 4 for service. At which time the 
write remote thread calls rem-send, the read remote 
thread unsuspends itself at step 52. The read remote 
thread binds itself to the remexd daemon at step 63. 45 
While input exists from rem-send in server 12 (FIG. 1). 
the remote thread continues to output data received 
from the server to the stdout until output file is empty. 
Then the read remote thread exits at step 65. 
[0058] Illustrated in FIG. 5 is the server processing in so 
the instance where default security is provided for 
remote procedure calls. Initialization of the server sys- 
tem is first performed at step 71. Next, the initialization 
of the remote execution server daemon is performed at 
step 72. The remote execution server daemon, more ss 
popularly known as "remexd." is a daemon that runs on 
the server systems 12, that listen for DCE remote pro- 
cedure calls from rem ex on the client systems 1 1 . 



[0059] Upon startup of the daemon (remexd) on the 
server system, it obtains the name of the single allowed 
client from a root-protected configuration fie, and is 
invoked with that client system as a parameter. Remexd 
performs the following actions. First, it registers its inter- 
face with the local rpcd (or deed) and invokes one of the 
registered procedures upon receipt of a call. Remexd 
also resets the stdin and stdout, and then forks and exe- 
cutes the remote send and receiv© procedures. 
[0060] Remexd is the remote end of the data pipeline 
described above in connection with remote execution 
client (remex). The remexd daemon manages the end 
point map or the server step code that communicates 
directly with the client subcode. The RPC daemon 
resides at well-known end points so the client can find it 
and communicate requests for service. In the initializa- 
tion, the remexd daemon registration of services pro- 
vided are performed. Also, registration of the protocol 
versions that the server will use for remote procedure 
call is performed. The protocol sequence identifies a 
single type of communication protocol, for example. 
TCP/IP or UDP/IP. Protocol registration causes the 
servers to create its end points. The clients get and use 
these end points to communicate with the server. The 
server then advertises its server location to clients by 
putting the end point information into a directly serviced 
database and storing the binding information into an 
application specific database and the creation of an end 
point map. Then, client applications can register the 
server address end points in a local end point map, thus 
enabling the clients stub code to search the end point 
map to get the proper addresses on the host and com- 
municate directly with them as discussed with regard to 
FIG. 1 . 

[0061 ] Next, the server system checks to see if DCE 
security is to be used in the remote procedure calls, and 
if this check for remote DCE security is false, then the 
server daemon sets authorization to the default security 
system at step 73. 

[0062] Next, the identification of the remexd client sys- 
tern is obtained in step 74. Registration of server serv- 
ices provided are performed at step 75. The server 
remexd is then suspended until a client binds to remexd 
to have services performed. Once the client binds to the 
remexd daemon, then the remexd daemon forks and 
executes a receive and send remote thread at step 76. 
The send remote thread, upon initialization, checks the 
security of the client at step 81. Remexd does the fol- 
lowing checks when using the default security system. 
[0063] First, it compares the source hostname with its 
knowledge of the registered server, (supplied as a com- 
mand line argument to remexd on startup). Next, it veri- 
fies that the sending user name is the official user. 
Remexd then checks that the pseudo-random number 
is not in a list of the last N that it has received. The 
pseudo-random number having been stored in a circu- 
lar array in the remexd memory space, each new one 
replacing the oldest number in the array. Remexd fur- 



6 



11 



EP0 947 925A2 



12 



ther constructs a string of the shared password (which 
is stored in an encoded fashion, so it can't be read using 
the strings command) and the pseudo-random number, 
and encrypts it using an encryption algorithm, and strips 
the first two characters. This result is compared with the 
encrypted string in the custom binding handle. If any of 
these checks fails, the call is rejected, and no session 
takes place. The failed attempt is loggedjn the security 
log, along with ail details.*.. 

[0064] The send remote thread next creates pipes at 
step 82. These pipes created at step 82 allow communi- 
cation from the send remote thread with the service pro- 
viding the program that actually provides the requested 
service. The program that provides the requested serv- 
ice is executed at step 83. While input exists from the 
client, the send remote thread write takes the data from 
the buffer received from the client process and writes it 
to the pipe created in step 82, thereby providing the 
data to the program that provides the requested service. 
[0065] Once the input from the client write remote 
thread ceases, the send remote thread closes the pipe 
to the service providing program, returns statuses to the 
write remote thread and exits the send remote thread at 
step 86. and returns to step 76 to wait for the next client 
to request service. 

[0066] The remote thread executed at step 77 initial- 
izes, and while output exists from the program that pro- 
vides the requested service, writes the data received 
from the program that provides the requested service 
and writes it to a buffer, this buffer is then returned to 
the client read remote thread in step 77. As long as 
there is data output from the program that provides the 
requested service, step 77 will loop until output from the 
program that provides the requested service ceases. 
[0067] Once the output from the program that pro- 
vides the requested service ceases, the receive remote 
thread closes the pipe to the program that provides the 
requested service and returns status to the client read 
remote thread of FIG. 4 in step 78. The receive remote 
thread then exits at step 79 and returns to a suspended 
state until the next client binds to the remexd daemon at 
step 76. 

[0068] Illustrated in FIG. 6 is the process for the client 
system security initialization and security process for 
remote procedure calls with DCE security. As the client 
system is initialized in step 91 . this includes operating 
system and all other initialization maintenance program 
loading. Step 91 also includes the execution of the client 
application program 21 (FIG. 2). 
[0069] Upon the client application program 21 execu- 
tion of a remote procedure call results in the initialization 
of the remex which is one per session in step 92. After 
initialization there is a check to see if the security for the 
remote procedure call is DCE. This is accomplished by 
the application checking, via a DCE API, whether it's in 
a configured DCE cell. If so, it then attempts to get a 
ticket using an internally-derived principal name. If 
these checks succeed, and it is able to verify its identity 
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with the security server, then the DCE security check is 
indicated as true. If this is true, then the authorization is 
set to the DCE security method at step 93. otherwise 
the process utilizes the method as shown in FIG. 6. 

5 [0070] Remex configures DCE Security by doing the 
following checks. First, it tries to get a current ceil name. 
Second, it tries to get a login identity. Then, there is an 
attempt to validate the identity from a keytabte. Next, 
rernex checks for an expired password and that remex 

10 got valid network credentials. Remex tries to certify the 
login identity, and sets the identity to be the current login 
context. If all the above works, then a global flag is set 
indicating the use of DCE Security. If the DCE security 
flag is set, then authentication is done on the first call of 

is both of the RPCs. The above checks require manual 
configuration of the client and the target hosts servers to 
be members of the cell and to have the proper principal 
names and keytables. 

[0071] Next, The client obtains its credentials for the 
20 security server 13 (FIG. 2) in step 94. The credentials 
are checked and the encryption key is updated in step 
95. 

[0072] The write and read remote threads are created 
and executed with the fork command at step 96. The 
25 write remote and read remote threadsexecute in parallel 
and after completion of the remote procedure call, each 
of the write remote and read remote threadsare termi- 
nated. 

[0073] The write remote thread, upon execution, first 

30 suspends the read remote thread, sets up the authori- 
zation information and binds to the remexd daemon of 
the server system at step 101. Next, the write remote 
thread reads from stdin at step 1 02. 
[0074] The write remote thread performs a remote 

35 procedure call to remsend in the server 1 2 system (FIG. 
2) and unsuspends the read remote thread at step 103. 
At this time, the read remote thread is unsuspended and 
continues execution, which will be described in later 
paragraphs. The write remote thread performs reading 

40 of the stdin file and remote procedure calling of the rem- 
sent procedure until input from stdin file is empty at step 
104. Once input from stdin is empty, then the write 
remote thread exits at step 105. 
[0075] Upon execution of the read remote thread, the 

45 read remote thread suspends until the first rem-send 
call is sent to the server at step 111. This is done 
because until there is a remote procedure call to the 
server, there can be no returning of data from this 
server based on the remote procedure call for service. 

so At which time the write remote thread calls rem-send. 
the read remote thread unsuspends itself at step 1 12. 
. [0076] The read remote thread sets up the authoriza- 
tion information and binds itself to the remexd daemon 
at step 113. While input exists from rem-send in server 

55 12 (FIG. 2). the remote thread continues to output data 
received from the server to the stdout until output is 
empty. Then, the read remote thread exits at step 115. 
[0077] Illustrated in FIG. 7 is the server processing in 
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the instance where DCE security is provided for remote 
procedure calls. Initialization of the server system is first 
performed at step 121. Next, the initialization of the 
remexd daemon is performed at step 122. 
[0078] The server system then checks to see if DCE 
security is to be used in the remote procedure calls, and 
if this check for remote DCE security is true, then the 
server daemon sets authorization to the DCE security 
system at step 123. Upon initialization, if remexd finds 
itself in a DCE cell, it checks for membership and iden- 
tity following the same sequence as described above for 
remex. Upon receiving the first RPC of a session, 
authorization is done in the reference monitor function 
in DCE terms. The reference monitor uses DCE, to 
check for the expected level of protection and client prin- 
cipal name. In the DCE mode, authentication is per- 
formed for each RPC at the stub level. The server 
system get its credentials from the security server 13 
(FIG. 2) instep 124. 

[0079] Registration of server services provided are 
performed at step 125. The remexd daemon manages 
the end point map or the server step code that commu- 
nicates directly with the client subcode. The RPC dae- 
mon resides at well-known end points so the client can 
find H and communicate requests for service. 
[0080] In the initialization, the remexd daemon regis- 
tration of services provided are performed. Also, regis- 
tration of the protocol seasons that the server will use 
for remote procedure calls is also performed. The proto- 
col sequence identifies a single type of communication 
protocol, for exanple but not limited to, TCP/IP or 
UDP/IR Protocol registration causes the servers to cre- 
ate its end points. The clients get and use these end 
points to communicate with the server. 
[0081] The server then advertises its server location 
to clients by putting the end point information into a 
directly serviced database, storing the binding informa- 
tion into an application specific database and the crea- 
tion of an end point map. Then, client applications can 
register the server address end points in a local end 
point map. thus enabling the clients stub code to search 
the end point map to get the proper addresses on the 
host and communicate directly with them as discussed 
with regard to FIG. 2. 

[0082] The server remexd is then suspended until a 
client binds to a remexd to have services performed at 
step 127. Once the client binds to the remexd daemon, 
the remexd daemon forks and executes a receive and 
send remote thread at step 127. 
[0083] The send remote thread, upon initialization, 
checks the authorization of the client at step 131. The 
send remote thread next return the authorization at step 
132. 

[0084] The send remote thread then creates pipes at 
step 133. These pipes created at step 133 allow com- 
munication from the send remote thread with the serv- 
ice providing the program that actually provides the 
requested service. The program that provides the 



requested service is executed at step 134. 
[0085] While input exists from the cHent, the send 
remote thread write takes the data from the buffer 
received from the client process and writes it to the pipe 

5 created in step 135. thereby providing the data to the 
program that provides the requested service. Once the 
input from the client write remote thread ceases, the 
send remote thread closes the pipe to the service pro- 
viding program, returns statuses to the write remote 

w thread at step 1 36, exits the send remote thread at step 
1 37, and returns to step 127 to wait for the next client to 
request service. 

[0086] The remote thread executed at step 1 41 initial- 
izes, and while output exists from the program that pro- 

15 vides the requested service, checks the authorization of 
the call from the client read remote thread, and writes 
the data received from the program that provides the 
requested service to a buffer. This buffer is then 
returned to the client read remote thread in step 141 . As 

20 long as there is data output from the program that pro- 
vides the requested service, step 141 will loop until out- 
put from the program that provides the requested 
service ceases. 

[0087] Once the output from the program that pro- 
25 vides the requested service ceases, the receive remote 
thread closes the pipe to the program that provides the 
requested service and returns status to the client read 
remote thread of FIG. 6 in step 142. The receive remote 
thread then exits at step 143 and returns to a sus- 
30 pended state, until the next client binds to the remexd 
daemon at step 127. 

[0088] The foregoing description has been presented 
for purposes of illustration and description. It is not 
intended to be exhaustive or to limit the invention to the 

35 precise forms disclosed. Obvious modifications or vari- 
ations are possible in light of the above teachings. The 
embodiment or embodiments discussed were chosen 
and described to provide the best illustration of the prin- 
ciples of the invention and its practical application to 

40 thereby enable one of ordinary skill in the art to utilize 
the invention in various embodiments and with various 
modifications as are suited to the particular use contem- 
plated. All such modifications and variations are within 
the scope of the invention as determined by the 

45 appended claims when interpreted in accordance with 
the breadth to which they are fairly and legally entitled. 

Claims 

so 1- An apparatus for remotely exiecuting commands 
using remote procedure calls (RPCs) in a distrib- 
uted computing environment (DCE), comprising: 

a client computer device (1 1); 
55 a server computer device (1 2); and 

a mechanism (38) for dynamically adapting 
security methods in the DCE by sensing if DCE 
security can be utilized between the client (1 1) 
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and the server ( 12) devices. providing a single system-level authorization 

file that is root-protected; and 
2.. The apparatus of claim 1 , wherein said dynamically providing jimits to client access to a single sys- 

adapting security mechanism further comprising: tern. 

logic (38) configured for establishing the default 
security structure (73) when the default secu- 

■ - • ritifjijaltad 1 isio .be'ulHizad. 

3. The apparatus of claim 1 , wherein sard dynamically 10 
adapting security mechanism further comprising: 

logic (38) configured for authorizing the DCE 
security (132) when the DCE security method 
is to be utilized. is 

4. The apparatus of claim 1 , wherein said dynamically 
adapting security mechanism further comprising: 

logic (38) configured to provide a shared secret 20 
key. 

5. The apparatus of claim 1 , wherein said dynamically 
adapting security mechanism further comprising: 

25 

logic (38) configured to provide a single sys- 
tem-level authorization file that is root-pro- 
tected; and 

logic (38) configured to provide limits to client 
access to a single system. so 

6. A method for use in a computer system for remotely 
executing commands using remote procedure calls 
(RPCs) in the distributed computing environment 
(DCE), the method comprising the steps of: 35 

sensing if DCE security is to be applied as sys- 
tem security; and 

utilizing the sensed system security in commu- 
nications between the client and the server 40 
devices. 

7. The method of claim 6, further including the step of: 

establishing the default security structure when as 
the default security method is to be utilized. 

8. The method of claim 6. further including the step of: 

authorizing the DCE security when the DCE so 
security method is to be utilized. 

9. The method of claim 6, further including the step of: 

providing a shared secret key. 55 

1 0. The method of claim 6, further including the step of : 
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